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1J.  AWnUCT  (Mt^mumiOQwcna) 

Our  research  is  concerned  with  "high  level"  vision  with  a  strong  biological  slant. 

The  last  two  or  three  decades  have  seen  breathtaking  progress  in  the  three  disciplines 
—  cognitive  psychology,  AI  and  visual  neurophysiology  —  but  they  have  been  pursued 
more  or  less  independently.  Wo  believe  that  the  time  is  now  ripe  for  forging  links 
between  these  disciplines  for  an  integrated  approach  to  vision.  We  have  had  two  goals 
in  mind;  11)  To  developiconceptual  links  between  neurophysiology  and  perception;  (2) 

To  develop  specific  tests  for  computational  models  of  human  vision.  Our  research  has 
called  into  question  several  widely  accepted  dogmas  concerning  the  mechanisms  of 
early  vision.  Alsol  we  have  been  able  to  discover  several  novel  visual  phenomena 
(e.g.,  motion  capture,  stereo-capture,  etc.)  and  have  identified  a  wide  range  of  new 
"natural  constraints"  that  govern  the  perception  of  shape-from  shading  structure  from 
motion  and  motion  correspondence.  Also,  we  have  discovered  striking  perceptual 
correlates  of  several  welI-knov,-n  physiological  observations  (e.g.,  "phantom  contours"-- 
j  stimuli  which  selectively  activate  the  magnocellular  pathway. 
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Our  research  is  concerned  with  "high  level"  vision  with  a  strong  biological  slant.  The  last 
two  or  three  decades  have  seen  breathtaking  progress  in  the  three  disciplines  --  cognitive 
psychology,  AI  and  visual  neurophysiology  --  but  they  have  been  pursued  more  or  less 
independently.  We  believe  that  the  bme  is  now  ripe  for  forging  links  between  these  disciplines 
for  an  integrated  approach  to  vision. 

We  have  had  two  goals  in  mind: 

1 .  To  develop  conceptual  links  between  neurophysiology  and  perception; 

2.  To  develop  specific  tests  for  computational  models  of  human  vision. 

While  it  is  true  that  some  of  our  work  has  called  into  question  whole  classes  of  "models" 
they  have  also  inspired  several  new  computational  models  (e.g.,  models  of  "motion  capture" 
proposed  by  Bulthoff,  Yuille.  Koch  and  others). 

During  the  last  decade  the  work  of  David  Marr  (1981)  has  had  a  tremendous  impact  on 
our  field  --  mostly  positive.  Indeed,  his  work  has  revolutionized  the  study  of  vision  in  a  manner 
analogous  to  the  Chomskyan  revolution  in  linguistics.  There  arc,  however,  several  major  pitfalls 
associated  with  his  approach.  Consider  the  four  basic  pillars  on  which  Marr's  edifice  rests: 

1.  Complex  information  processing  systems  --  such  as  human  vision  --  can  be 
studied  at  different  'levels"  --  i.e.,  the  abstract  level  of  the  "computational  problem,"  the  level  of 
algorithm  and  the  level  of  hardware  implementation.  Marr  urges  us  not  to  get  "confused" 
between  these  levels  --  they  must  be  kept  quite  separate  if  we  wish  to  avoid  getting  into 
conceptual  muddles. 

2.  The  most  important  critical  step  in  understanding  any  visual  process  is  to  clearly 
formulate  the  "computational  problem"  preferably  in  formal  mathematical  language. 

3.  "Segmentation"  of  the  visual  image  is  a  complex,  ill  po.'^ed,  and  largely  intractable 
problem  in  AI.  Fortunately,  however,  many  of  the  processes  of  early  vision,  e.g.,  stereopsis, 
motion  correspondence,  structure  from  motion,  shape  from  shading,  etc.,  take  place  prior  to 
image  segmentation.  In  fact  their  output  can  lead  to  segmentation  (e.g.,  Julesz'  stereograms). 
Therefore  we  can  study  and  model  these  processes  without  worrying  too  much  about 
segmentation. 

4.  "Top  down"  processes  based  on  high  level  semantics  have  no  role  in  human 

vision. 


These  four  assumptions  seem  reasonable  enough  at  first  glance,  but  our  work  suggests 
that  none  of  them  really  holds  up  on  careful  scrutiny.  Let  us  consider  each  in  turn. 

1 .  The  argument  about  levels  may  be  valid  as  a  logical  point  (one  recalls  Gilbert 
Ryles  remarks  about  "category  mistakes")  but  from  a  strategic  point  of  view  the  advice  is 
misguided.  In  our  view  the  only  sure  way  to  progress  in  understanding  any  biological 
information  processing  system  --  such  as  vision  -  is  to  develop  conceptual  links  between  levels 
instead  of  t^ing  to  keep  them  separate.  As  an  analogy  consider  the  manner  in  which  our 
understanding  of  the  double  helical  structure  of  the  DNA  molecule  (i.e.,  the  "hardware") 
completely  transformed  our  understanding  of  classical  heredity  and  genetics  which,  until  then, 
had  remained  a  "black  box"  subject.  There  is  now  a  wealth  of  empirical  evidence  that  the  same 


principle  holds  for  understanding  human  vision,  i.e.,  the  neural  machinery  in  our  visual  pathways 
powerfully  constraints  our  perceptual  experience  of  the  world  (e.g.,  Ramachandran  &  Gregory, 
1978;  Ramachandran,  1987;  Ramachandran,  1991;  Ramachandran  &  Gregory.  1991;  Rogers- 
Ramachandran,  Ramachandran,  1991,  ARVO).  For  example,  we  have  now  devised  a  stimulus 
that  seems  to  selectively  activate  a  "fast"  sign-invariant  contour  processing  system  in  human 
vision  that  might  correspond  to  the  "magno  cellular  pathway"  of  physiologists. 

2.  Understanding  the  "computational  problem"  is  certainly  important,  as  emphasized 
by  Marr,  but  it  is  very  easy  to  prejudge  y^hat  the  problem  actually  is  unless  you  do  experiments, 
e.g.,  what  is  the  computational  goal  of  color  vision?  Also  "computational  problems"  such  as 
stereo  correspondence,  structure  from  motion  and  the  aperture  problem  were  first  identified  by 
doing  experiments  (e.g.,  by  Julesz,  Wallach  and  others)  and  they  were  not  deduced  from  first 
principles. 

3.  Work  done  in  our  lab  contradicts  Marr’s  claim  that  segmentation  docs  not 
influence  early  vision  modules.  What  we  find  in  fact  is  that  image  segmentation  produced  by 
cues  such  as  implied  occlusion,  for  exanripic,  can  powerfully  constrain  the  solution  to  many  early 
vision  problems  such  as  motion  correspondence  (Ramachandran,  1985;  Ramachandran,  1991) 
stcrcopsis  (Ramachandran,  1986),  structure  from  motion  (Ramachandran,  (2obb  &  Rogers- 
Ramachandran,  1986);  and  shape  from  shading  (Ramachandran,  1988).  Any  program  of 
research  on  vision  must  take  these  facts  into  account. 

For  example,  we  have  done  several  experiments  which  suggest  that  even  illusory 
contours  (defined  by  implied  occlusion)  can  profoundly  influence  the  processing  of  stereopsis, 
apparent  motion  and  shape  from  shading. 

4.  The  view  that  "top-down"  processes  play  no  role  in  human  vision  is  contradicted 
by  the  simple  observation  that  hollow  masks  do  not  look  hollow,  they  look  convex.  This  is  true 
even  when  the  visual  system  has  to  override  stereoscopic  disparity  (Helmholtz,  Gregory,  1976) 
or  the  assumption  of  overhead  lighting  (Ramachandran,  198b). 

One  could  argue,  however,  that  this  tendency  has  nothing  to  do  w  ith  familiarity  with 
faces.  The  illusion  may  ariss  from  a  generic  assumption  about  me  convt  k  ny  of  objects  (e.g., 
Hoffman,  )  rather  than  familiarity  with  faces.  To  test  this,  we  recently  tried  comparing  an 
upside-down  hollow  mask  with  a  hollow  mask  held  upright.  By  walking  away  from  the  mask 
until  it  jiua  reversed  (i.e.,  was  seen  as  convex),  we  found  that  larger  disparities  can  be  overridden 
by  the  latter  than  by  the  former.  Since  the  masks  are  otherwise  completely  identical,  the 
observed  difference  has  to  be  a  result  of  the  fact  that  upright  faces  are  more  "face-like"  than 
upside-down  ones.  Thus,  while  it  is  largely  true  that  early  vision  is  relatively  immune  from 
semantics  --  our  experiments  suggest  that  Marr  has  clearly  overstated  his  case. 

In  summary,  our  research  has  called  into  question  several  widely  accepted  dogmas 
concerning  the  mechanisms  of  early  vision.  Also,  we  have  been  able  to  discover  several  novel 
visual  phenomena  (e.g.,  motion  capture,  stereo-capture,  etc.)  and  have  identified  a  wide  range  of 
new  "natural  constraints"  that  govern  the  perception  of  shape-from  shading  (Ramachandran, 
1988;  Kleffncr  &  Ramachandran,  1992),  structure  from  motion  and  motion  correspondence. 
Also,  we  have  discovered  striking  perceptual  correlates  of  several  well-known  physiological 
observations  (e.g.,  "phantom  contours"  -  stimuli  which  selectively  activate  the  magnocellular 
pathway:  "filling  in'^  of  scotomas  described  by  Gilbert  &  Wiesel,  Gaines  &  others;  plasticity  of 
cortical  topography,  described  by  Mcrzenich,  Pons  and  Gatas). 

The  enclosed  reprints  provide  more  detailed  descriptions  of  research  that  we  have  been 
doing  along  these  lines. 
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Books 

1.  Appointed  Editor-jn-Chicf  of  a  four  volume  Encyclopedia  of  human  behavior.  Academic 
Press. 

2.  Two  book  contacts  for  the  Scientific  American  Library  series  -  one  on  the  human  brain 
with  Patricia  Churchland  and  the  other  on  "seeing." 

Invited  lectures,  appointments,  colloquiua  etc. 

1.  Invited  to  give  a  public  lecture  at  the  250th  anniversary  celebration  of  the  University  of 
Pennsylvania  School  of  Medicine, 

2.  Invited  speaker  at  the  annual  meeting  of  the  Neuroscicnccs  research  program  (NRP)  held 
at  the  Rockefeller  Institute  (March,  1991). 

3.  Invited  "keynote"  speaker  at  the  SPE  meeting,  San  Diego  (1992). 

4.  Invited  "keynote '  speaker  at  SIGRAPH,  1992,  held  in  Chicago. 

5.  Invited  speaker  at  special  symposium  on  "Neuronal  Group  Selection"  at  the  Rockefeller 
Institute  (NRP),  May,  1992,  organized  by  Max  Cowan  and  Gerald  Edelman. 

6.  Kenneth  Craik  lecture  given  at  Cambridge  University. 

7.  Colloqium  given  at  MIT. 

8.  Colloquium  given  at  Oxford  University. 

9.  Awarded  "Certificate  of  Appreciation  for  Outstanding  Contributions  to  Visual  Science" 
by  the  Optometric  Association  of  America. 

10.  Appointed  McDornel  Pew  Visiting  Fellow,  Oxford  University,  England. 

11.  Interviewed  on  BBC  television  ("Antenna").  Debate  with  Daniel  C.  Dennett.  Aired  on 
August  8, 1992. 

12.  Interviewed  on  PBS  television  (KC!ET,  LA)  for  a  program  entitled  "Inside  Information" 
which  aired  nationally. 
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